Managing augmented reality content associated with a physical location

ABSTRACT

The present disclosure relates to managing augmented reality content created on a first electronic device and viewed at a second electronic device. In some embodiments, the first electronic device determines its physical location, receives input representing user-generated augmented reality content, displays an augmented reality environment including the user-generated augmented reality content overlaid on a live view of the physical location, and sends the user-generated augmented reality content to an external storage repository. The second electronic device can then receive the user-generated augmented reality content, determine whether it is at the physical location, and display the user-generated augmented reality content when it is at the physical location.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/026,855, entitled “MANAGING AUGMENTED REALITY CONTENT ASSOCIATED WITHA PHYSICAL LOCATION,” filed Sep. 21, 2020, which is a continuation ofU.S. patent application Ser. No. 16/140,268, entitled “MANAGINGAUGMENTED REALITY CONTENT ASSOCIATED WITH A PHYSICAL LOCATION,” filed onSep. 24, 2018, which claims the benefit of U.S. Provisional PatentApplication No. 62/566,173, entitled “MANAGING AUGMENTED REALITY CONTENTASSOCIATED WITH A PHYSICAL LOCATION,” filed on Sep. 29, 2017, which arehereby incorporated by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates generally to augmented realityenvironments, and more specifically to managing augmented realitycontent.

BACKGROUND

A physical location refers to a physical environment or a location in aphysical environment. Physical locations can include physical objects,such as physical trees, physical buildings, and physical people.Augmented reality is a technology that allows virtual objects to becombined with the physical location and/or physical objects.

SUMMARY

Described herein are techniques for managing augmented reality contentcreated on a first electronic device and viewed at a second electronicdevice. In some embodiments, a technique at the first electronic deviceincludes determining, using a first location sensor, a physical locationof the first electronic device; receiving input representinguser-generated augmented reality content; displaying a first augmentedreality environment comprising the user-generated augmented realitycontent overlaid on a live view of the physical location; and sendingthe user-generated augmented reality content to an external storagerepository. The technique at the second electronic device includesreceiving the user-generated augmented reality content; determining,using a second location sensor, whether the second electronic device isat the physical location; in accordance with a determination that thesecond electronic device is at the physical location, displaying asecond augmented reality environment comprising the user-generatedaugmented reality content overlaid on a live view of the physicallocation.

In some embodiments, the technique further includes determining thefirst electronic device is authorized to send the user-generatedaugmented reality content to the external storage repository. In someembodiments, the technique further includes providing authenticationinformation to the external storage repository, wherein theauthentication information authorizes the first electronic device tosend the user-generated augmented reality content to the externalstorage repository. In some embodiments, the user-generated augmentedreality content includes classification information. In someembodiments, the classification information comprises genre of content,rating of content, type of content, or a combination thereof.

In some embodiments, a system for managing augmented reality contentincludes a first electronic device including one or more firstprocessors and first memory storing one or more first programsconfigured to be executed by the one or more first processors; and asecond electronic device including one or more second processors andsecond memory storing one or more second programs configured to beexecuted by the one or more second processors. The one or more firstprograms include instructions for determining, using a first locationsensor, a physical location of the first electronic device; receivinginput representing user-generated augmented reality content; displayinga first augmented reality environment comprising the user-generatedaugmented reality content overlaid on a live view of the physicallocation; and sending the user-generated augmented reality content to anexternal storage repository. The one or more second programs includeinstructions for receiving the user-generated augmented reality content;determining, using a second location sensor, whether the secondelectronic device is at the physical location; in accordance with adetermination that the second electronic device is at the physicallocation, displaying a second augmented reality environment comprisingthe user-generated augmented reality content overlaid on a live view ofthe physical location.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following description, reference is made to the accompanyingdrawings which form a part thereof, and which illustrate severalembodiments of the present disclosure. It is understood that otherembodiments may be utilized and structural and operational changes maybe made without departing from the scope of the present disclosure. Theuse of the same reference symbols in different drawings indicatessimilar or identical items.

FIGS. 1A-1B depict exemplary systems for use in variouscomputer-generated reality technologies, including virtual reality andmixed reality.

FIG. 2 illustrates an embodiment of a system for managing augmentedreality content.

FIG. 3A illustrates an exemplary technique for generating augmentedreality content on a first device and sending the content to an externalstorage repository.

FIG. 3B illustrates an exemplary technique for displaying on a seconddevice the augmented reality content created on a first device.

FIG. 4A illustrates an exemplary technique for generating augmentedreality content.

FIG. 4B illustrates an exemplary technique for displaying augmentedreality content.

FIG. 5 illustrates an exemplary technique for obtaining augmentedreality content.

DETAILED DESCRIPTION

Various embodiments of electronic systems and techniques for using suchsystems in relation to various computer-generated reality technologies,including virtual reality and mixed reality (which incorporates sensoryinputs from a physical environment), are described. In particular, thepresent disclosure provides techniques for managing augmented realitycontent created on a first electronic device and viewed at a secondelectronic device. In some embodiments, the first electronic devicedetermines its physical location, generates augmented reality contentassociated with the physical location, and sends the augmented realitycontent to an external storage repository. The second electronic devicecan then retrieve the augmented reality content created by the firstdevice, and display the content when the second device is at the samephysical location where the first device generated the content.

A physical environment refers to a physical world that people can senseand/or interact with without aid of electronic systems. Physicalenvironments, such as a physical park, include physical articles, suchas physical trees, physical buildings, and physical people. People candirectly sense and/or interact with the physical environment, such asthrough sight, touch, hearing, taste, and smell.

In contrast, an extended reality (XR) environment refers to a wholly orpartially simulated environment that people sense and/or interact withvia an electronic system. In XR, a subset of a person's physicalmotions, or representations thereof, are tracked, and, in response, oneor more characteristics of one or more virtual objects simulated in theXR environment are adjusted in a manner that comports with at least onelaw of physics. For example, a XR system may detect a person's headturning and, in response, adjust graphical content and an acoustic fieldpresented to the person in a manner similar to how such views and soundswould change in a physical environment. In some situations (e.g., foraccessibility reasons), adjustments to characteristic(s) of virtualobject(s) in a XR environment may be made in response to representationsof physical motions (e.g., vocal commands).

A person may sense and/or interact with a XR object using any one oftheir senses, including sight, sound, touch, taste, and smell. Forexample, a person may sense and/or interact with audio objects thatcreate 3D or spatial audio environment that provides the perception ofpoint audio sources in 3D space. In another example, audio objects mayenable audio transparency, which selectively incorporates ambient soundsfrom the physical environment with or without computer-generated audio.In some XR environments, a person may sense and/or interact only withaudio objects.

Examples of XR include virtual reality and mixed reality.

A virtual reality (VR) environment refers to a simulated environmentthat is designed to be based entirely on computer-generated sensoryinputs for one or more senses. A VR environment comprises a plurality ofvirtual objects with which a person may sense and/or interact. Forexample, computer-generated imagery of trees, buildings, and avatarsrepresenting people are examples of virtual objects. A person may senseand/or interact with virtual objects in the VR environment through asimulation of the person's presence within the computer-generatedenvironment, and/or through a simulation of a subset of the person'sphysical movements within the computer-generated environment.

In contrast to a VR environment, which is designed to be based entirelyon computer-generated sensory inputs, a mixed reality (MR) environmentrefers to a simulated environment that is designed to incorporatesensory inputs from the physical environment, or a representationthereof, in addition to including computer-generated sensory inputs(e.g., virtual objects). On a virtuality continuum, a mixed realityenvironment is anywhere between, but not including, a wholly physicalenvironment at one end and virtual reality environment at the other end.

In some MR environments, computer-generated sensory inputs may respondto changes in sensory inputs from the physical environment. Also, someelectronic systems for presenting an MR environment may track locationand/or orientation with respect to the physical environment to enablevirtual objects to interact with real objects (that is, physicalarticles from the physical environment or representations thereof). Forexample, a system may account for movements so that a virtual treeappears stationary with respect to the physical ground.

Examples of mixed realities include augmented reality and augmentedvirtuality.

An augmented reality (AR) environment refers to a simulated environmentin which one or more virtual objects are superimposed over a physicalenvironment, or a representation thereof. For example, an electronicsystem for presenting an AR environment may have a transparent ortranslucent display through which a person may directly view thephysical environment. The system may be configured to present virtualobjects on the transparent or translucent display, so that a person,using the system, perceives the virtual objects superimposed over thephysical environment. Alternatively, a system may have an opaque displayand one or more imaging sensors that capture images or video of thephysical environment, which are representations of the physicalenvironment. The system composites the images or video with virtualobjects, and presents the composition on the opaque display. A person,using the system, indirectly views the physical environment by way ofthe images or video of the physical environment, and perceives thevirtual objects superimposed over the physical environment. As usedherein, a video of the physical environment shown on an opaque displayis called “pass-through video,” meaning a system uses one or more imagesensor(s) to capture images of the physical environment, and uses thoseimages in presenting the AR environment on the opaque display. Furtheralternatively, a system may have a projection system that projectsvirtual objects into the physical environment, for example, as ahologram or on a physical surface, so that a person, using the system,perceives the virtual objects superimposed over the physicalenvironment.

An augmented reality environment also refers to a simulated environmentin which a representation of a physical environment is transformed bycomputer-generated sensory information. For example, in providingpass-through video, a system may transform one or more sensor images toimpose a select perspective (e.g., viewpoint) different than theperspective captured by the imaging sensors. As another example, arepresentation of a physical environment may be transformed bygraphically modifying (e.g., enlarging) portions thereof, such that themodified portion may be representative but not photorealistic versionsof the originally captured images. As a further example, arepresentation of a physical environment may be transformed bygraphically eliminating or obfuscating portions thereof.

An augmented virtuality (AV) environment refers to a simulatedenvironment in which a virtual or computer generated environmentincorporates one or more sensory inputs from the physical environment.The sensory inputs may be representations of one or more characteristicsof the physical environment. For example, an AV park may have virtualtrees and virtual buildings, but people with faces photorealisticallyreproduced from images taken of physical people. As another example, avirtual object may adopt a shape or color of a physical article imagedby one or more imaging sensors. As a further example, a virtual objectmay adopt shadows consistent with the position of the sun in thephysical environment.

There are many different types of electronic systems that enable aperson to sense and/or interact with various XR environments. Examplesinclude head mounted systems, projection-based systems, heads-updisplays (HUDs), vehicle windshields having integrated displaycapability, windows having integrated display capability, displaysformed as lenses designed to be placed on a person's eyes (e.g., similarto contact lenses), headphones/earphones, speaker arrays, input systems(e.g., wearable or handheld controllers with or without hapticfeedback), smartphones, tablets, and desktop/laptop computers. A headmounted system may have one or more speaker(s) and an integrated opaquedisplay. Alternatively, a head mounted system may be configured toaccept an external opaque display (e.g., a smartphone). The head mountedsystem may incorporate one or more imaging sensors to capture images orvideo of the physical environment, and/or one or more microphones tocapture audio of the physical environment. Rather than an opaquedisplay, a head mounted system may have a transparent or translucentdisplay. The transparent or translucent display may have a mediumthrough which light representative of images is directed to a person'seyes. The display may utilize digital light projection, OLEDs, LEDs,uLEDs, liquid crystal on silicon, laser scanning light source, or anycombination of these technologies. The medium may be an opticalwaveguide, a hologram medium, an optical combiner, an optical reflector,or any combination thereof. In one embodiment, the transparent ortranslucent display may be configured to become opaque selectively.Projection-based systems may employ retinal projection technology thatprojects graphical images onto a person's retina. Projection systemsalso may be configured to project virtual objects into the physicalenvironment, for example, as a hologram or on a physical surface.

FIG. 1A and FIG. 1B depict exemplary system 100 for use in variouscomputer-generated reality technologies, including virtual reality andmixed reality.

In some embodiments, as illustrated in FIG. 1A, system 100 includesdevice 100 a. Device 100 a includes various components, such asprocessor(s) 102, RF circuitry(ies) 104, memory(ies) 106, imagesensor(s) 108, orientation sensor(s) 110, microphone(s) 112, locationsensor(s) 116, speaker(s) 118, display(s) 120, and touch-sensitivesurface(s) 122. These components optionally communicate overcommunication bus(es) 150 of device 100 a.

In some embodiments, elements of system 100 are implemented in a basestation device (e.g., a computing device, such as a remote server,mobile device, or laptop) and other elements of the system 100 areimplemented in a head-mounted display (HMD) device designed to be wornby the user, where the HMD device is in communication with the basestation device. In some examples, device 100 a is implemented in a basestation device or a HMD device.

As illustrated in FIG. 1B, in some embodiments, system 100 includes two(or more) devices in communication, such as through a wired connectionor a wireless connection. First device 100 b (e.g., a base stationdevice) includes processor(s) 102, RF circuitry(ies) 104, andmemory(ies) 106. These components optionally communicate overcommunication bus(es) 150 of device 100 b. Second device 100 c (e.g., ahead-mounted device) includes various components, such as processor(s)102, RF circuitry(ies) 104, memory(ies) 106, image sensor(s) 108,orientation sensor(s) 110, microphone(s) 112, location sensor(s) 116,speaker(s) 118, display(s) 120, and touch-sensitive surface(s) 122.These components optionally communicate over communication bus(es) 150of device 100 c.

System 100 includes processor(s) 102 and memory(ies) 106. Processor(s)102 include one or more general processors, one or more graphicsprocessors, and/or one or more digital signal processors. In someembodiments, memory(ies) 106 are one or more non-transitorycomputer-readable storage mediums (e.g., flash memory, random accessmemory) that store computer-readable instructions configured to beexecuted by processor(s) 102 to perform the techniques described below.

System 100 includes RF circuitry(ies) 104. RF circuitry(ies) 104optionally include circuitry for communicating with electronic devices,networks, such as the Internet, intranets, and/or a wireless network,such as cellular networks and wireless local area networks (LANs). RFcircuitry(ies) 104 optionally includes circuitry for communicating usingnear-field communication and/or short-range communication, such asBluetooth®.

System 100 includes display(s) 120. In some examples, display(s) 120include a first display (e.g., a left eye display panel) and a seconddisplay (e.g., a right eye display panel), each display for displayingimages to a respective eye of the user. Corresponding images aresimultaneously displayed on the first display and the second display.Optionally, the corresponding images include the same virtual objectsand/or representations of the same physical objects from differentviewpoints, resulting in a parallax effect that provides a user with theillusion of depth of the objects on the displays. In some examples,display(s) 120 include a single display. Corresponding images aresimultaneously displayed on a first area and a second area of the singledisplay for each eye of the user. Optionally, the corresponding imagesinclude the same virtual objects and/or representations of the samephysical objects from different viewpoints, resulting in a parallaxeffect that provides a user with the illusion of depth of the objects onthe single display.

In some embodiments, system 100 includes touch-sensitive surface(s) 122for receiving user inputs, such as tap inputs and swipe inputs. In someexamples, display(s) 120 and touch-sensitive surface(s) 122 formtouch-sensitive display(s).

System 100 includes image sensor(s) 108. Image sensors(s) 108 optionallyinclude one or more visible light image sensor, such as charged coupleddevice (CCD) sensors, and/or complementary metal-oxide-semiconductor(CMOS) sensors operable to obtain images of physical objects from thereal environment. Image sensor(s) also optionally include one or moreinfrared (IR) sensor(s), such as a passive IR sensor or an active IRsensor, for detecting infrared light from the real environment. Forexample, an active IR sensor includes an IR emitter, such as an IR dotemitter, for emitting infrared light into the real environment. Imagesensor(s) 108 also optionally include one or more event camera(s)configured to capture movement of physical objects in the realenvironment. Image sensor(s) 108 also optionally include one or moredepth sensor(s) configured to detect the distance of physical objectsfrom system 100. In some examples, system 100 uses CCD sensors, eventcameras, and depth sensors in combination to detect the physicalenvironment around system 100. In some examples, image sensor(s) 108include a first image sensor and a second image sensor. The first imagesensor and the second image sensor are optionally configured to captureimages of physical objects in the real environment from two distinctperspectives. In some examples, system 100 uses image sensor(s) 108 toreceive user inputs, such as hand gestures. In some examples, system 100uses image sensor(s) 108 to detect the position and orientation ofsystem 100 and/or display(s) 120 in the real environment. For example,system 100 uses image sensor(s) 108 to track the position andorientation of display(s) 120 relative to one or more fixed objects inthe real environment.

In some embodiments, system 100 includes microphones(s) 112. System 100uses microphone(s) 112 to detect sound from the user and/or the realenvironment of the user. In some examples, microphone(s) 112 includes anarray of microphones (including a plurality of microphones) thatoptionally operate in tandem, such as to identify ambient noise or tolocate the source of sound in space of the real environment.

System 100 includes orientation sensor(s) 110 for detecting orientationand/or movement of system 100 and/or display(s) 120. For example, system100 uses orientation sensor(s) 110 to track changes in the positionand/or orientation of system 100 and/or display(s) 120, such as withrespect to physical objects in the real environment. Orientationsensor(s) 110 optionally include one or more gyroscopes and/or one ormore accelerometers.

FIG. 2 illustrates an embodiment of system 200 for managing augmentedreality content. System 200 includes first device 100 d, second device100 e, and storage repository 250. Devices 100 d and 100 e are eachembodiments of system 100, as described in reference to FIGS. 1A-1B.Storage repository 250 is a server or other remote computing device incommunication with devices 100 d and 100 e. Storage repository 250includes various components, such as processor(s) 252, networkingcircuitry(ies) 254, memory(ies) 256, and storage device(s) 258. Thesecomponents optionally communicate over communication bus(es) 260.

Processor(s) 252 include one or more general processors and/or one ormore digital signal processors. In some embodiments, memory(ies) 256 areone or more non-transitory computer-readable storage mediums (e.g.,flash memory, random access memory) that store computer-readableinstructions configured to be executed by processor(s) 252 to performthe techniques described below. Networking circuitry(ies) 254 includecircuitry for communicating with electronic devices (such as devices 100d and 100 e), networks, such as the Internet, intranets, and/or wirelessnetworks, such as cellular networks and wireless local area networks(LANs). Storage device(s) 258 include devices for storing and retrievingdigital information, such as hard disk drives, tape drives, andsolid-state drives.

In some embodiments of system 200, augmented reality content isgenerated on first device 100 d, sent to storage repository 250, andthen retrieved by second device 100 e for viewing. The augmented realitycontent generated on first device 100 d is associated with a particularphysical location. In some embodiments, first device 100 d determinesthe physical location using a global positioning system (GPS). In someembodiments, device 100 d determines the physical location by detectingphysical features present at the physical location (e.g., physicalobjects or features of physical objects, such as text identifying thelocation or landmarks associated with the location). First device 100 adetects the physical features by obtaining images of the physicallocation using one or more image sensor(s) (such as image sensor(s) 108described in reference to FIGS. 1A-1B).

First device 100 d generates the augmented reality content in responseto receiving input from a user. In some embodiments, first device 100 dreceives a selection of a virtual object for display, and an indicationof where the selected virtual object should be displayed in an augmentedreality environment. In some embodiments, the augmented reality contentis associated with a particular physical feature at the physicallocation, such as a wall of a building, a statue, a tree, and so on. Forexample, when first device 100 d receives user input selecting a virtualobject (e.g., a hat), first device 100 d displays the virtual objectoverlaying a live view of the physical location. The selected virtualobject is then placed on (or associated with) a particular physicalfeature at the physical location (e.g., the virtual hat appears on thehead of a statue at the physical location in the augmented realityenvironment). As another example, when first device 100 d receives userinput representing a virtual object (e.g., graffiti), first device 100 ddisplays the virtual object overlaying a live view of the physicallocation. The virtual object is then placed on (or associated with) aparticular physical feature at the physical location (e.g., virtualgraffiti appears on the wall of a building at the physical location inthe augmented reality environment).

After first device 100 d generates the augmented reality content, thecontent is sent to storage repository 250 where it is stored for laterretrieval by other devices (such as second device 100 e). The augmentedreality content stored at storage repository 250 includes the contentitself as well as an indication of the physical location and/or physicalfeature with which the content is associated. In some embodiments, theaugmented reality content includes classification information, such asgenre of content (e.g., art, history, games, etc.), rating of content(e.g., explicit or not explicit), and/or type of content (e.g.,authorized or unauthorized).

In some embodiments, prior to first device 100 d sending the augmentedreality content to storage repository 250, a determination is madewhether first device 100 d is authorized to send the content and/orassociate the content with the physical location. For example, aphysical location (such as a business) can limit the augmented realitycontent associated with its physical location to content from authorizeddevices. In some embodiments, first device 100 d is authorized byproviding authentication information to storage repository 250, such asa username and password.

In some embodiments, second device 100 e retrieves the augmented realitycontent from storage repository 250 when the location of second device100 e is determined to be at a physical location having associatedaugmented reality content (e.g., the physical location where firstdevice 100 d generated augmented reality content). In some embodiments,second device 100 d determines the location using a global positioningsystem (GPS). In some embodiments, device 100 d determines the physicallocation by detecting physical features present at the location (e.g.,text identifying the location or landmarks associated with thelocation). Second device 100 a detects the physical features byobtaining images of the location using one or more image sensor(s) (suchas image sensor(s) 108 described in reference to FIGS. 1A-1B).

In some embodiments, in response to a prediction that second device 100e will be located at a physical location in the future having associatedaugmented reality content (e.g., the physical location where firstdevice 100 d generated augmented reality content), second device 100 eidentifies the associated augmented reality content and retrieves thecontent from storage repository 250 prior to arriving at the physicallocation. In some embodiments, second device 100 e predicts its futurephysical location based on a destination received by second device 100e. For example, when second device 100 e includes navigation software(e.g., a map application) and receives a destination entered into thenavigation software, second device 100 e then utilizes the receiveddestination to predict a future physical location of the second device100 e. If the future physical location corresponds to a physicallocation having associated augmented reality content (e.g., the physicallocation where first device 100 d generated augmented reality content),then second device 100 e retrieves the augmented reality content beforearriving at the destination.

In some embodiments, second device 100 e determines a route to thereceived destination and identifies whether any additional augmentedreality content is available for locations along the route to thedestination. If additional augmented reality content is available, thensecond device 100 e retrieves the additional augmented reality contentfrom the storage repository 250 prior to arriving at the locations alongthe route.

In some embodiments, second device 100 e predicts its future physicallocation based on a direction of travel of second device 100 e. Forexample, when second device 100 e determines it is moving north, and thephysical location where first device 100 d generated augmented realitycontent is north of the current location of second device 100 e, thensecond device 100 retrieves the augmented reality content from thestorage repository 250. In some embodiments, second device 100 e detectsthe direction of travel using an orientation sensor (such as orientationsensor(s) 110 described in reference to FIGS. 1A-1B). In someembodiments, second device 100 e detects the direction of travel bydetermining changes in location as detected by GPS and/or by GPSguidance software.

In some embodiments, second device 100 e predicts its future physicallocation based on using other data sources accessible by second device100 e (e.g., calendar information or location history information). Forexample, when second device 100 e includes calendar information (such asfrom a calendar application) and the calendar information indicates thelocation and time of a scheduled appointment, second device 100 eutilizes the location and time of the appointment to predict a futurephysical location of the second device 100 e. If the future physicallocation corresponds to a physical location having associated augmentedreality content (e.g., the physical location where first device 100 dgenerated augmented reality content), then second device 100 e retrievesthe augmented reality content before arriving at the scheduledappointment. As another example, when second device 100 e includeslocation history information (such as from a map application), seconddevice 100 e predicts potential future physical locations based onprevious locations indicated by the location history information. Forexample, if the location history information indicates that seconddevice 100 e follows approximately the same route at approximately thesame time every weekday (such as when a user drives to work), thensecond device 100 e identifies whether any additional augmented realitycontent is available for locations along the route the second device 100e has a history of following. If additional augmented reality content isavailable, then second device 100 e retrieves the additional augmentedreality content from the storage repository 250.

In some embodiments, second device 100 e retrieves the augmented realitycontent when second device 100 e determines it is within a predefinedproximity to a physical location having associated augmented realitycontent (e.g., the physical location where first device 100 d generatedaugmented reality content). For example, when second device 100 edetermines it is within 0.5 miles of a physical location havingaugmented reality content, then second device 100 retrieves theaugmented reality content from the storage repository 250.

In some embodiments, when a variety of augmented reality contentassociated with a physical location is available, second device 100 efilters the available content based on user content preferences. Contentpreferences include genre of content (e.g., art, history, games, etc.),rating of content (e.g., explicit or not explicit), and/or type ofcontent (e.g., authorized or unauthorized).

In some embodiments, once the augmented reality content is retrieved andsecond device 100 e determines it is at the physical location associatedwith the content, second device 100 e displays an indication of theretrieved augmented reality content. In some embodiments, the indicationis an affordance indicating availability of the retrieved augmentedreality content for display. In some embodiments, second device 100 eprovides an alert (e.g., an audible alert and/or a tactile alert) whenthe retrieved augmented reality content is available for display.

In some embodiments, after displaying the indication and in response toreceiving a request to display the retrieved augmented reality content,second device 100 e displays the augmented reality content while seconddevice 100 e is at the associated physical location. Alternatively or inaddition, in some embodiments, once the augmented reality content isretrieved and second device 100 e determines it is at the physicallocation associated with the content, second device 100 e automaticallydisplays the augmented reality content. The second device 100 e displaysthe augmented reality content in an augmented reality environment byoverlaying the augmented reality content on a live view of the physicallocation. When the augmented reality content is associated with aparticular physical feature at the physical location, second device 100e displays the augmented reality content overlaying the physical featurein the augmented reality environment. For example, when the augmentedreality content includes a virtual object (e.g. a hat) associated with aphysical feature (e.g., a statue), second device 100 e displays thevirtual object overlaying a live view of the physical feature (e.g., thehat appears to be on the statue in the augmented reality environment).

FIG. 3A illustrates an embodiment of device 100 f displaying, on display120, augmented reality content 360. Device 100 f is an embodiment offirst device 100 d described in reference to FIG. 2. In FIG. 3A, device100 f is shown as a mobile device, such as a cellular phone. However, itshould be understood that device 100 f can be any device configured todisplay an augmented reality environment.

Device 100 f is displaying, on display 120, a representation 370 ofretail store 380 with augmented reality content 360. The representation370 of the retail store 380 includes representation 370 a of shelves 380a and representation 370 b of merchandise 380 b. Device 100 f overlaysaugmented reality content 360 on the representation 370 a of shelf 380a. Device 100 f tracks the location and/or orientation of shelf 380 awith respect to the position and/or orientation of device 100 f todisplay augmented reality content 360 as being on the representation 370a of shelf 380 a.

As shown in FIG. 3A, the augmented reality content 360 is a virtualgift. The virtual gift is created by device 100 f in response to a userselecting the virtual gift with device 100 f. Device 100 f also receivesan indication that the virtual gift should be displayed on therepresentation 370 a of shelf 380 a, which associates the virtual giftwith the physical shelf 380 a in the retail store 380.

After device 100 f generates the augmented reality content 360 (e.g.,the virtual gift), the content 360 is sent to a storage repository (suchas storage repository 250 described in reference to FIG. 2) where it isstored for later retrieval by other devices (such as second device 100 edescribed in reference to FIG. 2). The augmented reality content 360sent to the storage repository includes the content itself as well as anindication of the physical feature and location (e.g., the shelf 380 aof retail store 380) with which the content 360 is associated.

In some embodiments, prior to device 100 f sending the augmented realitycontent 360 to a storage repository, device 100 f providesauthentication information to the storage repository, such as a usernameand password, which authorizes device 100 f to send the augmentedreality content 360 and/or associate the content 360 with the physicalfeature and/or location. For example, a physical location (such as abusiness) can limit the augmented reality content associated with itsphysical location to content from authorized devices. For example,retail store 380 may require device 100 f to be an authorized device ofa store employee in order for the virtual gift to be associated withshelf 380 a in retail store 380.

FIG. 3B illustrates an embodiment of device 100 g displaying, on display120, the augmented reality content 360 created by device 100 f, asdescribed in reference to FIG. 3A. Device 100 g is an embodiment ofsecond device 100 e described in reference to FIG. 2. In FIG. 3B, device100 g is shown as a mobile device, such as an electronic tablet.However, it should be understood that device 100 g can be any deviceconfigured to display an augmented reality environment.

In some embodiments, device 100 g identifies the augmented realitycontent 360 (e.g., a virtual gift) and retrieves the content 360 from astorage repository (such as storage repository 250 described inreference to FIG. 2) when the location of device 100 g is determined tobe at retail store 380. In some embodiments, device 100 g retrieves theaugmented reality content 360 prior to arriving at retail store 380 inresponse to a prediction that device 100 g will be located at retailstore 380 in the future.

In some embodiments, once the augmented reality content 360 is retrievedand device 100 g determines it is proximate to shelf 380 a at retailstore 380, device 100 g displays an indication of the retrievedaugmented reality content 360, such as an affordance indicatingavailability of the augmented reality content 360 for display.Alternatively, in some embodiments, once the augmented reality content360 is retrieved and device 100 g captures images of shelf 380 a atretail store 380, device 100 g automatically displays augmented realitycontent 360 as being on representation 370 a of shelf 380 a.

FIG. 4A illustrates an exemplary technique 400 a for generatingaugmented reality content on a first device and sending the content toan external storage repository. In some embodiments, the technique iscarried out by system 200 described in reference to FIG. 2.

At block 402, a physical location of the first device is determined. Insome embodiments, the physical location is determined using GPS. In someembodiments, the physical location is determined by detecting physicalfeatures present at the physical location (e.g., text identifying thelocation or landmarks associated with the location). The physicalfeatures are detected by obtaining images of the physical location usingone or more image sensor(s) (such as image sensor(s) 108 described inreference to FIGS. 1A-1B).

At block 404, input is received representing user-generated augmentedreality content. In some embodiments, the augmented reality content isassociated with a particular physical feature at the physical location,such as a wall of a building, a statue, a tree, and so on.

At block 406, an augmented reality environment including theuser-generated augmented reality content is displayed. Theuser-generated augmented reality content is overlaid on a live view ofthe physical location.

In some embodiments, at block 408, authentication information isprovided to an external storage repository. At block 410, adetermination is made whether the authentication information authorizesthe first device to send the user-generated augmented reality content tothe external storage repository.

At block 412, the user-generated augmented reality content is sent tothe external storage repository. In some embodiments, the user-generatedaugmented reality content is sent to the external storage repository inaccordance with a determination that the first device is authorized atblock 410. In some embodiments, the user-generated augmented realitycontent includes classification information. The classificationinformation includes genre of content (e.g., art, history, games, etc.),rating of content (e.g., explicit or not explicit), and/or type ofcontent (e.g., authorized or unauthorized).

FIG. 4B illustrates an exemplary technique 300 b for displaying on asecond device the augmented reality content created on the first devicein technique 300 a. In some embodiments, the technique is carried out bysystem 200 described in reference to FIG. 2.

At block 414, user-generated augmented reality content created on thefirst device (e.g., at block 404 of FIG. 4A) is received. Theuser-generated augmented reality content is received from an externalstorage repository (e.g., storage repository 250 described in referenceto FIG. 2).

At block 416, a current location of the second device is determined. Insome embodiments, the current location is determined using GPS. In someembodiments, the current location is determined by detecting physicalfeatures present at the current location (e.g., text identifying thelocation or landmarks associated with the location). The physicalfeatures are detected by obtaining images of the current location usingone or more image sensor(s) (such as image sensor(s) 108 described inreference to FIGS. 1A-1B).

At block 418, a determination is made whether the current location ofthe second device is the physical location where the first devicecreated the augmented reality content.

At block 420, in accordance with a determination that the second deviceis at the physical location where the first device created the augmentedreality content, an augmented reality environment including theuser-generated augmented reality content created by the first device isdisplayed. The user-generated augmented reality content is overlaid on alive view of the physical location.

FIG. 5 illustrates an exemplary technique 500 for obtaining augmentedreality content. In some embodiments, the technique is carried out bysystem 200 described in reference to FIG. 2.

At block 502, a future physical location of an electronic device (e.g.,second device 100 e described in reference to FIG. 2) is predicted. Insome embodiments, the predicted future physical location is predictedbased on a received destination (e.g., a destination entered into GPSnavigation software). In some embodiments, predicting the futurephysical location includes determining a direction of travel of theelectronic device. In some embodiments, predicting the future physicallocation includes determining a proximity of the electronic device tothe future physical location.

At block 504, augmented reality content associated with the futurephysical location is identified. In some embodiments, the augmentedreality content includes virtual objects for display in an augmentedreality environment. In some embodiments, the identified augmentedreality content includes content associated with additional physicallocations along a route to a destination (e.g., a destination enteredinto GPS navigation software).

At block 506, before arriving at the future physical location, theidentified augmented reality content is obtained. In some embodiments,the identified augmented reality content is obtained from an externalstorage repository (e.g., storage repository 250 described in referenceto FIG. 2).

At block 508, a current location of the electronic device is determined.In some embodiments, the current location is determined using GPS. Insome embodiments, the current location is determined by detectingphysical features present at the current location (e.g., textidentifying the location or landmarks associated with the location). Thephysical features are detected by obtaining images of the currentlocation using one or more image sensor(s) (such as image sensor(s) 108described in reference to FIGS. 1A-1B).

At block 510, a determination is made whether the current location ofthe electronic device is the future physical location.

At block 512, in accordance with a determination that the electronicdevice is at the future physical location, an indication of the obtainedaugmented reality content is displayed. In some embodiments, displayingan indication of the obtained augmented reality content includesdisplaying an affordance indicating availability of the obtainedaugmented reality content for display. In some embodiments, an alert isprovided when the obtained augmented reality content is available fordisplay.

In some embodiments, at block 514, the augmented reality content isdisplayed while the electronic device is at the future physicallocation. In some embodiments, the augmented reality content isdisplayed in response to receiving a request to view the augmentedreality content. In some embodiments, the augmented reality content isautomatically displayed in response to the electronic device being atthe future physical location.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to provideusers with augmented reality content. The present disclosurecontemplates that in some instances, this gathered data may includepersonal information data that uniquely identifies or can be used tocontact or locate a specific person. Such personal information data caninclude demographic data, location-based data, telephone numbers, emailaddresses, twitter IDs, home addresses, data or records relating to auser's health or level of fitness (e.g., vital signs measurements,medication information, exercise information), date of birth, or anyother identifying or personal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used todeliver augmented reality content that is of greater interest to theuser. Further, other uses for personal information data that benefit theuser are also contemplated by the present disclosure. For instance,health and fitness data may be used to provide insights into a user'sgeneral wellness, or may be used as positive feedback to individualsusing technology to pursue wellness goals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof augmented reality content delivery services, the present technologycan be configured to allow users to select to “opt in” or “opt out” ofparticipation in the collection of personal information data duringregistration for services or anytime thereafter. In another example,users can select to limit the length of time personal information datais maintained. In addition to providing “opt in” and “opt out” options,the present disclosure contemplates providing notifications relating tothe access or use of personal information. For instance, a user may benotified upon downloading an app that their personal information datawill be accessed and then reminded again just before personalinformation data is accessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, augmentedreality content can be selected and delivered to users by inferringpreferences based on non-personal information data or a bare minimumamount of personal information, such as the content being requested bythe device associated with a user, other non-personal informationavailable to the augmented reality content delivery services, orpublicly available information.

While the present disclosure has been shown and described with referenceto the embodiments provided herein, it will be understood by thoseskilled in the art that various changes in form and details may be madewithout departing from the scope of the present disclosure.

What is claimed is:
 1. An electronic device, comprising: one or moreprocessors; and memory storing one or more programs configured to beexecuted by the one or more processors, the one or more programsincluding instructions for: predicting a future physical location of anelectronic device; identifying augmented reality content associated withthe future physical location, wherein the augmented reality content isassociated with the future physical location based on a previous userrequest to associate the augmented reality content with the futurephysical location; before the electronic device arrives at the futurephysical location, obtaining the identified augmented reality content;upon arriving at the future physical location, displaying an affordanceindicating availability of the obtained augmented reality content fordisplay; receiving a request to view the obtained augmented realitycontent; and in response to receiving the request to view the obtainedaugmented reality content, displaying the obtained augmented realitycontent.
 2. The electronic device of claim 1, wherein the one or moreprograms further include instructions for: obtaining images from one ormore image sensors, wherein the obtained images include one or morepredefined physical features present at the future physical location;and determining a current location of the electronic device based atleast in part on the obtained images.
 3. The electronic device of claim1, wherein the augmented reality content is displayed while theelectronic device is at the future physical location.
 4. The electronicdevice of claim 1, wherein the one or more programs further includeinstructions for: providing an alert when the obtained augmented realitycontent is available for display.
 5. The electronic device of claim 1,wherein the one or more programs further include instructions for:receiving a destination, wherein the predicted future location is basedon the received destination.
 6. The electronic device of claim 5,wherein the identified augmented reality content includes contentassociated with additional physical locations along a route to thedestination.
 7. The electronic device of claim 1, wherein predicting thefuture physical location comprises: determining a direction of travel ofthe electronic device.
 8. The electronic device of claim 1, whereinpredicting the future physical location comprises: determining aproximity of the electronic device to the future physical location.
 9. Anon-transitory computer-readable storage medium storing one or moreprograms configured to be executed by one or more processors, the one ormore programs including instructions for: predicting a future physicallocation of the electronic device; identifying augmented reality contentassociated with the future physical location, wherein the augmentedreality content is associated with the future physical location based ona previous user request to associate the augmented reality content withthe future physical location; before the electronic device arrives atthe future physical location, obtaining the identified augmented realitycontent; upon arriving at the future physical location, displaying anaffordance indicating availability of the obtained augmented realitycontent for display; receiving a request to view the obtained augmentedreality content; and in response to receiving the request to view theobtained augmented reality content, displaying the obtained augmentedreality content.
 10. The non-transitory computer-readable storage mediumof claim 9, wherein the one or more programs further includeinstructions for: obtaining images from one or more image sensors,wherein the obtained images include one or more predefined physicalfeatures present at the future physical location; and determining acurrent location of the electronic device based at least in part on theobtained images.
 11. The non-transitory computer-readable storage mediumof claim 9, wherein the augmented reality content is displayed while theelectronic device is at the future physical location.
 12. Thenon-transitory computer-readable storage medium of claim 9, wherein theone or more programs further include instructions for: providing analert when the obtained augmented reality content is available fordisplay.
 13. The non-transitory computer-readable storage medium ofclaim 9, wherein the one or more programs further include instructionsfor: receiving a destination, wherein the predicted future location isbased on the received destination.
 14. The non-transitorycomputer-readable storage medium of claim 13, wherein the identifiedaugmented reality content includes content associated with additionalphysical locations along a route to the destination.
 15. Thenon-transitory computer-readable storage medium of claim 9, whereinpredicting the future physical location comprises: determining adirection of travel of the electronic device.
 16. The non-transitorycomputer-readable storage medium of claim 9, wherein predicting thefuture physical location comprises: determining a proximity of theelectronic device to the future physical location.
 17. A method,comprising: predicting a future physical location of the electronicdevice; identifying augmented reality content associated with the futurephysical location, wherein the augmented reality content is associatedwith the future physical location based on a previous user request toassociate the augmented reality content with the future physicallocation; before the electronic device arrives at the future physicallocation, obtaining the identified augmented reality content; uponarriving at the future physical location, displaying an affordanceindicating availability of the obtained augmented reality content fordisplay; receiving a request to view the obtained augmented realitycontent; and in response to receiving the request to view the obtainedaugmented reality content, displaying the obtained augmented realitycontent.
 18. The method of claim 17, further comprising: obtainingimages from one or more image sensors, wherein the obtained imagesinclude one or more predefined physical features present at the futurephysical location; and determining a current location of the electronicdevice based at least in part on the obtained images.
 19. The method ofclaim 17, wherein the augmented reality content is displayed while theelectronic device is at the future physical location.
 20. The method ofclaim 17, further comprising: receiving a destination, wherein thepredicted future location is based on the received destination.
 21. Themethod of claim 17, wherein predicting the future physical locationcomprises: determining a direction of travel of the electronic device.22. The method of claim 17, wherein predicting the future physicallocation comprises: determining a proximity of the electronic device tothe future physical location.